Summary

Note: We do not expect this to get merged, instead, we want to showcase what we need to make Viceroy work for local integration testing and local developer experience.

At Stellate, we're in need of an integration test harness and more flexible local developer experience compared to what is currently possible with the Viceroy lib. This PR shows what parts of Viceroy we expanded with new capabilities in order for it to be powerful enough to build a testing framework around it.

• Adds the ability to read Endpoint (log sink) messages programmatically by having them write to an in-memory buffer that can be read from outside of the execution context after an invocation. Technically this could have been intercepted from stdout before, but this is much more convenient.
• Adds the ability to handle requests in-memory, instead of reaching out to an origin via HTTP. This is done by adding an in-memory handler to Backends to which Requests will be passed.
• Adds the ability to intercept dynamic backend registrations and traffic, which in combination with in-memory backend handlers now allows for in-memory handling of dynamic backend requests.
• Cache API support. Allows keeping states in between invocations. No transaction support, plus some calls unused by Stellate are not yet implemented. Very simplistic cache implementation that is not really suited for long-running processes, but works for us.
• Adds the ability to wait for guest code invocations to fully complete, which is necessary to surface execution errors after an initial response has been sent to client (especially true if streams are processed and panics happen after the headers have been sent back to the client already).

Ideally, these changes (or at least comparable changes) make it into Viceroy so we don't have to maintain a fork. We're happy to have a discussion what needs to be done, talk through ideas, walk through code and give you access to our testing code so you can directly see and play around with how we envision Viceroy to function in an integration test setup.

Note that we'll record a video on the ins and outs / requirements and we'll also reach out to you directly via Slack with a document to start a discussion.

Implementation notes

• CLI code is neither looked at nor touched beyond what was necessary to get it to compile and probably doesn't work in this branch as intended.
• We could implement service chains with these changes. However, our focus is on testing a single service.
• In-memory backends are super basic and don't perform any checks. It would be worth expanding upon the concept.
• Endpoint listeners are also super basic in concept and execution, they probably require a second look as well.
• Caches use a lot of lock unwraps.